Modular sensing device implementing state machine gesture interpretation

ABSTRACT

A modular sensing device can include an inertial measurement unit to generate sensor data corresponding to user gestures performed by a user, a mode selector enabling the user to select a mode of the modular sensing device out of a plurality of modes, and one or more output devices to generate output based on the user gestures and the selected mode. The modular sensing device can further include a controller to implement a plurality of state machines. Each state machine can be associated with a corresponding user gesture by a sensor data signature. The state machine can execute a state transition when the sensor data matches the sensor data signature. The executed state transition can cause the controller to generate a corresponding output via the one or more output devices specific to the selected mode and based on the corresponding user gesture.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/253,799, entitled “Modular Sensing Device Implementing State MachineGesture Interpretation,” filed on Aug. 31, 2016, which applicationclaims the benefit of and priority to U.S. Provisional PatentApplication No. 62/274,514, entitled “Portable Sensing Device forProcessing Gestures as Input,” filed on Jan. 4, 2016, and U.S.Provisional Application No. 62/346,216, entitled “Modular Sensing Devicefor Processing Gestures as Input,” filed on Jun. 6, 2016; theaforementioned priority applications being hereby incorporated byreference in their respective entireties.

BACKGROUND

Remotely operated self-propelled devices are typically operable by wayof analog or digital controller devices that communicate a limitedamount of preconfigured commands. Such commands typically involvesignaled radio frequency communications to accelerate and maneuver theself-propelled device. Furthermore, wearable device technology inconsumer electronics is rapidly being integrated into routine useractivities, such as sporting activities, content viewing or browsing,and task-oriented activities (e.g., gaming).

Furthermore, wireless networks typically utilize protocols that enablewireless devices to detect signal sources from other devices forinitiating data and communication links. Such networks are typicallyimplemented using networking hardware, which may be incorporated invarious wireless network devices, such as access points (APs),peer-to-peer (P2P) devices, wireless local area network (LAN) equippeddevices, and the like—each advertising a unique identity (e.g., a mediaaccess control (MAC) address) indiscriminately to devices within range.Connections may be established with such devices to transmit and receivedata.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure herein is illustrated by way of example, and not by wayof limitation, in the figures of the accompanying drawings in which likereference numerals refer to similar elements, and in which:

FIG. 1A illustrates a block diagram of an example modular sensing devicefor use in controlling a remotely operated device, according to examplesdescribed herein;

FIG. 1B is a block diagram illustrating an example modular sensingdevice, as shown and described herein;

FIG. 2A is a flow chart describing an example method of generatingcommands by a modular sensing device, in accordance with examplesdescribed herein;

FIG. 2B is a flow chart describing a state machine-based example methodof controlling a remotely operated device using a modular sensingdevice;

FIG. 3 illustrates an example modular sensing device insertable into aplurality of compatible apparatuses;

FIG. 4 illustrates an embodiment of multiple sensing devices thatconcurrently provide input for a program or application which utilizesthe inputs, along with inferences which can be made about a person orobject that carries the devices, according to one or more examples;

FIG. 5 illustrates a system which concurrently utilizes input frommultiple modular sensing devices in connection with execution of anapplication or program; and

FIG. 6 is a block diagram of an example modular sensing device uponwhich examples described herein may be implemented.

DETAILED DESCRIPTION

Examples described herein relate to a multi-modal modular sensingdevice, worn or carried by a user (e.g., as a wrist-worn device), toenable a variety of interactions with other devices through sensemovement of the modular sensing device. Among other activities, examplesprovide for a modular sensing device that can individually, or incombination with another device (e.g., controller device, such as amobile computing device) control other devices, interact with compatibledevices of other users, and/or operate in connection with task-orientedactivities (e.g., gameplay). In some examples, the modular sensingdevice corresponds to a wearable device (e.g., a watch, a pendant for anecklace, a hat, glasses) can be placed in multiple modes to, forexample, control the characteristics of movement of another device, orinteract with the real-word. For example, the modular sensing device cancontrol acceleration and maneuvering of a self-propelled device.

In certain aspects, a modular sensing device can include an inertialmeasurement unit (IMU), and can be operated in multiple modes in whichgestures (e.g., arm gestures) may be translated based on the particularmode of the wearable device. In one example, the modular device can beinsertable into multiple devices, such as a wrist worn device, clothing,an accessory device (e.g., a toy sabre or sword), a wearable pendant,ring, hat, and the like. The modular sensing device can further includeoutput devices, such as lighting elements (e.g., one or more LEDs), anaudio device (e.g., a speaker), and/or a haptic feedback system. In oneexample, the modular sensing device further includes a charging port(e.g., a mini-universal serial bus port) to enable charging of a powersource included in the device.

According to some examples, a modular sensing device is operable todetect its own movement in three-dimensional space, using the IMU. Insome implementations, the IMU can be an integrated device.Alternatively, the IMU can be implemented through a combination ofsensors, such as a three-dimensional accelerometer or gyroscope. Incertain examples, the modular sensing device can include a processor andmemory to interpret the sensor data, and to communicate interpretedsensor data to another device (e.g., mobile computing device) using awireless connection (e.g., BLUETOOTH). In variations, the sensor datacan generate data that is either raw data or processed, depending on theprocessing resources that are selected for the modular sensing device,and the processing load which is implemented for the modular sensingdevice.

In many implementations, the modular sensing device can generate output(e.g., haptic, audio, and/or visual output) or control commands foroperating a remote controlled device by utilizing state machines inmemory that correspond to individual gestures or combinations ofgestures. Depending on a selected mode of the modular sensing device,processing resources of the modular sensing device can monitor logicalstate machines or automatons that correspond to specified sensor datacombinations (e.g., based on gestures performed by a user). For example,a single state machine can correspond to an initial arm raising gestureperformed by the user wearing a wrist-worn device having the modularsensing device. A sensor profile from a single sensor or multiplesensors included in the IMU (e.g., gyroscopic sensor and/oraccelerometer) can indicate the initial arm raising gesture, which cantrigger a state transition for the state machine. This state transitioncan cause processing resources of the modular sensing device toautomatically generate a control command to cause the remote controlleddevice to accelerate.

Additionally, a second state machine can correspond to an increasedangle in the arm raising gesture. When the sensor profile of the IMUindicates the gesture above a certain predetermined angle, a statetransition can be triggered for the second state machine, which cantrigger an increased acceleration command to be generated automatically.Several state machines can be comprised in memory of the modular sensingdevice, and can each correlate to a specified sensor data profile, fromsimple single-action gestures, to complex multiple-motion gestures.Furthermore, an interpretation engine (e.g., a processor) can interpreteach state transition for each state machine based on a particular modeand/or sub-mode of the modular sensing device to generate acorresponding output (e.g., a control command and/or a combination ofvisual, haptic, and audible output).

As used herein, a “modular sensing device” can comprise an electronicdevice that includes sensor resources for detecting its own movement,and of dimension and form factor suitable for being insertable into anynumber of devices configured for receiving the device. Numerous examplesof modular sensing devices are provided in the context of a “wearabledevice,” such as a wrist worn device (e.g., watch, watch band,bracelet). But as noted by other examples, variations to the type andform factor of a wearable device can vary significantly, encompassing,for example, eyeglasses, hats, pendants, armbands, glasses and variousother form factors. While many examples describe functionality in thecontext of a wearable device, embodiments extend such examples to otherform factors in which a modular sensing device may be inserted, such aswands, fobs, wielded toys, or mobile communication devices.

In many examples, the wearable device can include one or more sensors todetect the device's own movements. In particular, a wearable deviceincludes an accelerometer and/or a gyroscopic sensor. In some examples,sensor data, corresponding to user gestures performed by the userwearing the wearable device, can be translated into control commands ordata packets to be transmitted and implemented based on the selectedmode of the wearable device. According to many examples, the wearabledevice can include an inductive interface to inductively pair with otherdevices, which can trigger a specified mode on the wearable device. Forexample, an inductive pairing between the wearable device and aself-propelled device can trigger a drive mode in which the wearabledevice can be utilized by the user to operate the self-propelled device.Additionally or alternatively, the wearable device can include an inputmechanism, such as an analog or digital button, that enables the user toselect a particular mode and/or scroll through a series of modes for thewearable device.

Among other functionality, some examples described herein provide foralternative modes of operation, including, for example (i) a “drivemode” in which the wearable device is utilized to control aself-propelled device; (ii) a “control mode” in which the wearabledevice is utilized in connection with smart home devices; (iii) a“finding mode” or “finder mode” in which the wearable device is utilizedto detect virtual or digital resources; (iv) a “mining mode” which canbe initiated by the user to collect virtual resources when they aredetected in the finder mode; (v) a “training mode” in which the wearabledevice is utilized in connection with a self-propelled device to assistthe user in training for certain achievements or for increasing theuser's abilities to perform task-oriented activities (e.g., increasingskills for a subsequent game or sporting activity); (vi) a “sword mode”in which the wearable device provides feedback (e.g., haptic, audio,and/or visual feedback) when the user performs actions while holding anobject; (vii) a “default mode” in which the device monitors for anddetects other proximate wearable devices (e.g., wearable devices of thesame type) which enables the users to pair with each other's wearabledevices; (viii) a “interactive mode” or “battle mode” selectable inresponse to two or more device pairings in which users are able tointeract with each other with predetermined sets of actions (e.g.,offensive and defensive actions learned and perfected by userspracticing in the training mode); (ix) a “sharing mode” selectable inresponse to two or more device pairings in which users can shareinformation stored in each other's wearable devices, or user accountsassociated with the wearable devices (e.g., sharing collected virtualresources discovered and mined in the finder and mining modes to beexpended or consumed in a gameplay environment); and (x) a “gaming mode”in which the wearable device can be utilized in connection with a game.

Still further, numerous examples make reference to a “self-propelled”device. A self-propelled device can include, for example, a device thatcan be wirelessly and remotely controlled in its movement, whether themovement is on ground, water or air. For example, a self-propelleddevice can include a wirelessly and remotely controlled drone, car,airplane, helicopter, boat, etc. While conventional examples enablecontrol of a self-propelled device, conventional approaches generallyutilize a perspective of the device being controlled. While someconventional devices, for example, enable a computing device held by theuser to project a perspective of the device under control, examplesdescribed herein enable control of such devices to utilize anorientation of the user. Specifically, some examples include a modularsensing device that can determine an orientation of the user, andfurther enable control of the self-propelled device through anenvironment that accommodates or is in the perspective of the user,based on the orientation of the user (as determined by the modularsensing device). By way of example, the control of a self-propelleddevice can be projected through an orientation or perspective of theuser for purpose of a virtual environment.

Some examples include a modular sensing device having a wirelesscommunication module (e.g., a BLUETOOTH low energy module) that enablescommunication of sensor data (e.g., raw sensor data from theaccelerometer or gyroscopic sensor), or translated data (i.e.,translations of the sensor data based on the selected mode of thewearable device). In some examples, the sensor data may be relayed fortranslation by a mobile computing device before being transmitted toanother device (e.g., a paired wearable device or a pairedself-propelled device). In other examples, processing resources of thewearable device can execute mode instructions, based on the selectedmode, to translate the sensor data (e.g., via use of state machines thattrigger control commands or feedback based on a selected mode of themodular sensing device) for direct transmission to one or more otherdevices, as described herein.

As used herein, a “user gesture” includes gestures performed by a userwhile utilizing the modular sensing device, whether inserted into awearable device or a wielded object. For example, the modular sensingdevice may be inserted into a wrist-worn device, in which case gesturesmay be any number of physical movements or actions that affect the IMUof the device when it is worn on the wrist. Such movements and actionscan include shaking, arm movements (e.g., raising lowering, pointing,twisting, and any combination thereof), wrist movements, hand actions(such as grasping or grabbing), and the like. However, wearable devicesdescribed herein are not limited to wrist-worn devices, but may beutilized as a ring (e.g., a finger-worn device), an ankle-worn device, aneck-worn device, a head-worn device, a belt (e.g., a waist-worndevice), etc. Thus, gestures performed using the modular sensing devicecan be any actions or movements in which correlated sensor data fromsensors of the device can be translated into commands, instructions,feedback, etc. depending on the mode of the wearable device.

Among other benefits, examples described herein achieve a technicaleffect of enhancing user interactivity with other devices and otherusers. Such interactivity may include utilizing the modular sensingdevice to control a self-propelled device, interact with other users ofwearable devices, collect and share data, control smart home devices,interact with the real-world via a gaming interface through the modularsensing device, and the like.

One or more examples described herein provide that methods, techniques,and actions performed by a computing device, a modular sensing device,or a self-propelled device are performed programmatically, or as acomputer-implemented method. Programmatically, as used herein, meansthrough the use of code or computer-executable instructions. Theseinstructions can be stored in one or more memory resources of thecomputing device. A programmatically performed step may or may not beautomatic.

One or more examples described herein can be implemented usingprogrammatic modules or components of a system. A programmatic module orcomponent can include a program, a sub-routine, a portion of a program,or a software component or a hardware component capable of performingone or more stated tasks or functions. As used herein, a module orcomponent can exist on a hardware component independently of othermodules or components. Alternatively, a module or component can be ashared element or process of other modules, programs or machines.

Some examples described herein can generally require the use ofcomputing devices, including processing and memory resources. Forexample, one or more examples described herein can be implemented, inwhole or in part, on computing devices such as digital cameras, digitalcamcorders, desktop computers, virtual reality (VR) or augmented reality(AR) headsets, cellular or smart phones, personal digital assistants(PDAs), laptop computers, printers, digital picture frames, and tabletdevices. Memory, processing, and network resources may all be used inconnection with the establishment, use, or performance of any exampledescribed herein (including with the performance of any method or withthe implementation of any system).

Furthermore, one or more examples described herein may be implementedthrough the use of instructions that are executable by one or moreprocessors. These instructions may be carried on a computer-readablemedium. Machines shown or described with figures below provide examplesof processing resources and computer-readable mediums on whichinstructions for implementing examples can be carried and/or executed.In particular, the numerous machines shown with examples includeprocessor(s) and various forms of memory for holding data andinstructions. Examples of computer-readable mediums include permanentmemory storage devices, such as hard drives on personal computers orservers. Other examples of computer storage mediums include portablestorage units, such as CD or DVD units, flash memory (such as carried onsmart phones, multifunctional devices or tablets), and magnetic memory.Computers, terminals, network enabled devices (e.g., mobile devices,such as cell phones) are all examples of machines and devices thatutilize processors, memory, and instructions stored on computer-readablemediums. Additionally, examples may be implemented in the form ofcomputer-programs, or a non-transitory computer usable carrier mediumcapable of carrying such a program.

System Description

FIG. 1A illustrates a block diagram of an example modular sensing devicefor use in controlling a remotely operated device, according to examplesdescribed herein. The modular sensing device 100 shown in FIG. 1A may beintegrated with a wearable device 102 (e.g., a wrist-worn device,pendant, clothing, a hat, eyeglasses, an ankle-worn device, a ring,etc.), or can be inserted into a compartment or otherwise included as apart of the wearable device 102. The wearable device 102, or modularsensing device 100 disposed therein, can include a mode selector 110,such as an analog button that enables the user to select a mode of thedevice. In one example, repeated user input 111 on the mode selector 110enables the user to scroll through a list of available modes. Such modescan include, but are not limited to a drive mode, a control mode, afinder mode, a mining mode, a training mode, a sword mode, a defaultmode, an interactive or battle mode, a sharing mode, and a gaming mode,as described herein. As shown in the block diagram of FIG. 1A, a userinput 111 on the mode selector 110 can cause a processor 120 of themodular sensing device 100 to generate an output 132 that indicates theparticular mode selected. For example, the modular sensing device 100can include a number of output devices 130, such as an audio device, ahaptic system, and/or visual elements (e.g., an LED array or display).Each user input 111 on the mode selector 110 can trigger an audio,haptic, and/or visual output 132, indicating the particular mode (e.g.,a drive mode). As an example, the output 132 can comprise a voice outputthat states the selected mode, or a combination of voice, visual, andhaptic output.

Additionally or alternatively, the user may connect the wearable device102 with a mobile computing device, such as the user's smart phone ortablet computer. Mode selection may be performed automatically by theuser initiating a designated application of the mobile computing device,such as a smart home application, a controller application (e.g., tocontrol a self-propelled device), a gaming application, and the like. Invariations, the user can execute a designated application in connectionwith the wearable device 102 that enables the user to scroll through thevarious modes. The user may scroll through the modes on the mobilecomputing device, or via successive selection inputs on the modeselector 110, which can trigger the mobile computing device to display aselectable mode. In other variations, multiple types of inputs can beperformed in the mode selector 110, such as tap gestures and tap andhold gestures, which can correlate to scrolling through the modes andselecting a particular mode respectively. As provided herein, the modeselector 110 can be an input mechanism such as an analog or digitalbutton, a touch panel such as a track pad, or a miniaturetouch-sensitive display device.

According to examples provided herein, the modular sensing device 100can include a memory 115 that stores mode instruction sets executable bythe processor 120 based on the mode selected by the user. Each modeinstruction set can cause the processor 120 to interpret sensor data 137in a particular manner. Along these lines, same or similar gestures 106performed by the user can result in different generated outputs 132 orcommands 108 depending on the mode selected by the user. In someimplementations, selection of a particular mode can cause the processor120 to initiate a communications module 125 of the modular sensingdevice 100 to establish a wireless connection with another device. Inone example, the modular sensing device 100 can include a BLUETOOTH lowenergy module to enable communications with one or more other devices,such as a second modular sensing device or a remotely operated device140.

The modular sensing device 100 can be relatively small in size comparedto current computing devices, and in many implementations, the modularsensing device 100 does not include a power-intensive memory andcomputational resources. In such implementations, the memory can becomprised of a series of state machines that, when transitioned, cantrigger a particular output automatically. Further description of thestate machine implementations is provided below with respect to FIG. 1B.Furthermore, examples described herein with respect to the drive mode ofthe modular sensing device 100 can also be implemented in the statemachine examples described herein.

As provided herein, the memory 115 can include a drive mode instructionset 117 executable by the processor 120 in response to a user input 111on the mode selector 110. In some aspects, execution of the drive modeinstructions 117 can cause the processor 120 to initiate thecommunications module 125 to establish a communications link 104 with aproximate remotely operated device 140. In variations, the modularsensing device 100 can include an induction interface 127 which cantrigger the communication link 104 between the modular sensing device100 and the remotely operated device 140. For example, upon selectingthe drive mode, the user can place the wearable device 102 withininductive range (e.g., ˜2-5 cm) of the remotely operated device 140,which can include a corresponding induction interface. The inductivetransfer of communication information can enable the modular sensingdevice 100 to establish the communication link accordingly.

According to examples described, the modular sensing device 100 canfurther include an inertial measurement unit (IMU) 135 that can comprisea gyroscope and an accelerometer for accurate measurement of the modularsensing device's 100 movement and orientation. In variations, the IMU135 can further include a magnetometer to, for example, assist incalibration based on the orientation. Once the communication link 104 isestablished in the drive mode, the processor 120 can monitor the sensordata 137 for the particular gestures 106 being performed by the user. Inseveral examples, the gestures 106 can correspond to the user raising orlowering an arm, and/or performing additional arm motions. The sensordata 137 from the IMU 135 can comprise movement, position, and/ororientation information that the processor 120 can interpret inaccordance with the drive mode. For example, gestures 106 performed bythe user can be detected by the processor 120 via sensor data 137 fromthe IMU 135. Each of the gestures 106 can be interpreted by theprocessor 120 as one or more control commands 108 to be executed by theremotely operated device 140.

In one example, the drive mode can be automatically initiated inresponse to a particular detected gesture, regardless of the currentmode of the modular sensing device 100. This gesture can correspond to adistinct sensor data signature that, when detected by the processor 120in executing any mode, overrides that mode and initiates the drive modeautomatically. Thus, upon detection of the distinct sensor datasignature, the processor 120 can automatically initiate thecommunications module 125, establish the communications link 104 withthe remotely operated device 140, and generate control commands 108based on the detected gestures 106 performed by the user in the sensordata 137. The modular sensing device 100 may then dynamically transmitsuch control commands 108 to the remotely operated device 108 forexecution as they are generated by the processor 120. In one example,the specific gesture corresponds to a pushing motion with an arm wearingthe wearable device 102 performed by the user. As provided herein, thispushing motion can correspond to a specified sensor data signature notused for any other mode, and is therefore distinct to enable theprocessor 120 to identify it irrespective of the current mode of themodular sensing device 100.

In certain examples, gestures 106 such as raising an arm can cause theprocessor 120 to generate acceleration commands to drive away from theuser. Lowering the arm can cause the processor to generate decelerationand/or reverse commands. Further, moving the arm from side to side cancause the processor 120 to generate steering or directional commands.For example, moving the arm left can cause the remotely operated device140 to turn left, and moving the arm right can cause the device 140 toturn right as the device 140 travels away from the user. Such controlcommands 108 may be processed by a controller of the remotely operateddevice 140, or may be directly executed on the drive motors of thedevice 140 in order to accelerate and maneuver the device 140 inaccordance with the gestures 106 performed by the user.

Furthermore, in the drive mode, angular thresholds can be established inthe drive mode instructions 117 that can determine the manner in whichthe processor 120 interprets the sensor data 137. When such thresholdsare crossed, the processor 120 can alter interpretation of the sensordata 137 into alternative commands 108. For example, as the user raisesthe arm above an angular threshold (e.g., 45 degrees), and/or changes anorientation of the arm (e.g., palm down to palm up), the processor 120can alter the interpretation of the sensor data 137 such that remotelyoperated device 140 drives towards the user as the arm is raised.Furthermore, in such a state, the directional interpretation of thesensor data 137 can be reversed such that moving the arm left can causethe remotely operated device 140 to turn right, and moving the arm rightcan cause the remotely operated device 140 to turn left. Thisdirectional reversal triggered by the angular threshold, and incombination with the change in orientation of the user's arm, can createa palm control illusion of the remotely operated device 140 by the user.Thus, in the drive mode, specified gestures detected in the sensor data137 (e.g., the user's arm rotating or crossing an angular threshold) cantrigger the processor 120 to interpret the sensor data 137 differentlyin order to produce the illusion.

FIG. 1B is a block diagram illustrating an example modular sensingdevice, as shown and described herein. As provided herein, the modularsensing device 150 can be space limited, and can only include a limitedamount memory and computational resources. In such implementations, themodular sensing device 150 can represent each possible gesture that canbe performed by a user as a state machine. Thus, for each gesture, astate machine corresponding to that gesture can either positivelyidentify its gesture or negatively identify its gesture. When a positivegesture is identified, the state machine triggers a state transitionwhich can cause an output generator 160 to generate a particular outputaccordingly. As provided herein, the output generator 160 and memory 180can comprise a hardware controller implementing the various statemachines to generate the output commands 164.

As an illustration, the modular sensing device 150 can include a memory180 implementing a number of state machines (e.g., SM1 181, SM2 183, SM3185, SM4 187, . . . , SMN 189), each being associated with a particulargesture. For example, SM1 181 can be associated with the user raising anarm, SM2 183 can be associated with the user lowering an arm, SM3 185can be associated with the user pointing an arm to the right, and SM4187 can be associated with the user pointing an arm to the left.Furthermore, any number of state machines may be implemented in thememory 210 representing any number of gestures. At any given time step,the state machines can be instantiated for each gesture type, and eachstate machine can continuously inspect the instantaneous sensor data 167from the IMU 165 in order to initialize or instantiate, transitionindividual states along a state string, terminate a current statestring, or trigger an accept state or final state. When the final stateis triggered, this means that the particular gesture corresponding tothat state machine has been performed by the user.

According to examples, each state machine can consist of a finite set ofstates (a fixed string of states), one or more inputs, one or moreoutputs, a transition function, and an initial state. The state machinecan be linked to a particular gesture by way of a sensor data signature,which can comprise an accelerometer data signature, gyroscope datasignature, or a combination of both. Furthermore, the state machine canbe linear and directional, with each state having a particular errortolerance in its sensor data signature. A final state of a state machinecan thus be triggered when the full sensor data signature of aparticular gesture is matched to the sensor data 167 generated by theIMU 165.

In some aspects, if at any time after instantiating, an associatedgesture corresponding to a respective state machine is not beingperformed, the input string for that state machine, and for thatparticular instantiation, automatically terminates. At any giveninstance, the state machine either terminates from the outset (e.g., aninitial aspect of the sensor data signature for the gesture is notmatched), or instantiates the input string towards the final state. Atany given state along the input string, the state machine can terminateif the gesture being performed diverges from the error tolerances builtinto the state machine. If, however, each state along the input stringis transitioned accordingly (i.e., the sensor data 167 from the IMU 165matches the sensor data signature for that state machine within itserror tolerances), the final state is triggered for that state machine.

The memory 180 can include a state machine reporter 182 that can reportsuch final state transitions 188 to an output generator 160 of themodular sensing device 150. The output generator 160 can be configuredbased on the particular mode of the modular sensing device 150.Accordingly, final state reports 188 from individual state machines canbe interpreted differently depending on the mode. In other words, theoutput from the modular sensing device 150 for a particular gesture(e.g., a backhanded swinging motion) can be different depending on themode initiated via the mode selector 155. Furthermore, certain finalstate reports 188 from the state machine reporter 182 can correspond tosub-mode triggers 186 for the particular mode. Such sub-mode triggers186 may not trigger an output, but rather trigger the output generator160 to alter interpretation of certain final state reports 188 in orderto generate a alternative output.

Such outputs generated by the output generator 160 can comprise controlcommands 162 to operate a remotely operated device, such asacceleration, steering, and deceleration commands. In some aspects, theoutput generator 160 can generate output commands 164 for the modularsensing device's 150 haptic system 192, visual system 194, and/or audiosystem 196. Thus, as final state reports 188 are received, the outputgenerator 160 can cause the haptic 192, visual 194, and audio 196systems of the modular sensing device 150 to produce individual orcombined outputs. As described herein, such outputs can includevibration feedback or guidance, colored lights or display output, tonaloutputs such as audible chimes that indicate positive and/or negativefeedback, and voice output.

In the example of the drive mode described with respect to FIG. 1A, eachcompleted arm gesture can correspond to a final state machine report188, and can cause the output generator 160 to generate a controlcommand 162 accordingly. In the example shown in FIG. 1B, the userraising an arm above a certain angle (e.g., five degrees) can cause astate machine corresponding to that gesture to transition to a finalstate. The state machine reporter 182 can report the final state 188 ofthat state machine to the output generator 160, which, based on thedrive mode interpretation, can generate a control command 162 toinitiate acceleration of the remotely operated device. This controlcommand 162 may then be transmitted to the remotely operated device viathe communications module 175 of the modular sensing device 150.

Referring to both FIGS. 1A and 1B, in certain modes, the communicationsmodule 125, 175 can act as a signal or beacon sensor which can providesignal data to the processor 120 or output generator 160. Based on themode, the processor 120 or output generator 160 can generate a specifiedresponse based on the signal data. For example, the signal data canindicate a unique identifier of the signal source. The memory 115, 180can include a UID log of specified signal sources that allow the user tointeract with such sources based on the mode of the modular sensingdevice 100, 150. For example, if the signal source corresponds toanother modular sensing device, the user can be enabled to select a modein which the user can interact with the other user (e.g., play aphysical game involving gestures).

As described with respect to FIG. 1A, the interpretation of the sensordata 137 can be altered based on a particular gesture trigger, such asthe user's arm exceeding an angular threshold. The analog with respectto FIG. 1B comprises a state machine in the memory 180 that correlatesto the particular threshold gesture. Thus, when the sensor data 167indicates the specified threshold gesture, the correlated state machinecan transition to its final state, which can be reported to the outputgenerator 160 accordingly. Such a final state report 188 can comprise asub-mode trigger 186, in that the sub-mode trigger 186 causes the outputgenerator 160 to alter interpretation of certain final state reports 188while remaining in the same mode. In the drive mode example, a sub-modetrigger 186 can correspond to the user raising an arm above a certainthreshold (e.g., 45 degrees), which can cause the output generator 160to, for example, reverse the interpretation of the final state reports188 corresponding to the directional commands for the remotely operateddevice.

Additionally, each mode of the modular sensing device 150 can includeany number of sub-mode triggers 186 that cause the output generator 160to alter an interpretation of or disregard a particular final statereport 188 corresponding to a specified sensor data signature. Forexample, the drive mode can include a set of angular gesture thresholds(e.g., raising an arm beyond 45 degrees, lower the arm below 45 degrees,turning the arm from palm down to palm up). A state machine can bededicated—within the specified mode—to a sensor data signatureindicating the user gesture crossing a gesture threshold. Thus, when theuser gesture crosses the gesture threshold, the dedicated state machinecan transition to its final state, which, when reported to the outputgenerator 160 (i.e., as a sub-mode trigger 186), can cause the outputgenerator 160 to alter interpretation of certain final state reports 188within that mode. In some aspects, disregarded final state reports priorto the sub-mode trigger can now trigger a specified output (e.g., anaudio, haptic, and/or visual output, or a particular control command162).

In further examples, when the modular sensing device 150 is in aparticular mode, a specific complex gesture—represented by a particularstate machine in the memory 180—can cause the output generator 160 toreconfigure its interpretation of certain final state reports 188,execute a sub-mode, or automatically initialize a different mode for themodular sensing device 150. Accordingly, for any given instance, sensordata 167 from the IMU 165 can continuously cause the various statemachines to instantiate, terminate, instigate state transitions, and/ortransition to a final state. In aspects described, only when a statemachine transitions to its final state does the output generator 160generate output commands 164 and/or control commands 162 to providefeedback, operative control over a remotely operated device, guidancevia the output devices, and/or task-based instructions (e.g., inaccordance with a particular game).

Furthermore, the various aspects performed by the modular sensing device100 described with respect to FIG. 1A may also be performed by examplemodular sensing devices 150 as shown and described with respect to FIG.1B. Thus, the execution of designated modal instruction sets by themodular sensing device 100 of FIG. 1A—in which the processor 120directly interprets the sensor data 137 based on the executinginstruction set—may be substituted by the state machine examples asdescribed with respect to FIG. 1B. In other words, the limited memoryand computational resources of the modular sensing device 150 of FIG. 1Bmay be compensated by attributing sensor data signatures to statemachines, requiring less memory and processing without sacrificingfunctionality.

Methodology

FIG. 2A is a flow chart describing an example method of generatingoutput commands by a modular sensing device, in accordance with examplesdescribed herein. In the examples described with respect to FIG. 2A,reference may be made to reference characters representing like featuresas shown and described with respect to FIGS. 1A and 1B. Furthermore, themethods and processes described with respect to FIG. 2A may be performedby an example modular sensing device 100, 150 as shown and describedwith respect to FIGS. 1A and 1B. Referring to FIG. 2A, the modularsensing device 150 can receive a mode selection input (200). In someexamples, the mode selection input can comprise the user physicallypushing an analog mode selector 155 on the modular sensing device 150.Additionally or alternatively, the mode selection input can comprise aspecified gesture performed while using the modular sensing device 150.

In response to the mode selection input, the modular sensing device 150can interpret the final state machine reports 188 in accordance with themode selected (205). As provided herein, the modular sensing device 150can operate in any number of modes, with each mode corresponding tocontrolling a remote device (e.g., drive mode), generating user tasksand feedback in connection with a remote device (e.g., training mode),generating user tasks and feedback in conjunction with another modularsensing device (e.g., playing a real-world game with another user),and/or generating standalone user tasks and feedback. As providedherein, user tasks can comprise instructions or suggestions to the uservia the output devices of the modular sensing device in accordance withthe selected mode. Such instructions or suggestions can be generatedbased on a programmed game, in which the user is to utilize the modularsensing device 150 to perform gestures and action, search for a certainsignal source, cause the remotely operated device 140 to perform a setof actions, and the like. Furthermore, feedback can comprise reactiveoutput to the tasked actions to be performed by the user. For example,the feedback can comprise audio, visual, and/or haptic responses toactions indicating affirmative or negative completion of such tasks. Inone example, the feedback indicates to the user whether an instructedtask or gesture (e.g., indicated by the sensor data 167 and correlatedstate machine) has successfully been performed.

According to several examples, the final state machine reports 188 canbe correlated to a specified output. Thus, based on each final statemachine report 188 the modular sensing device 150 can generate commandsin accordance with the mode and, when relevant, the sub-mode of thedevice 150 (220). As provided herein, such generated commands caninclude output commands 164 to output audio (222), visual (226), and/orhaptic (224) output on the modular sensing device 150. Additionally oralternatively, the modular sensing device 150 can generate controlcommands 162 (228) for operating a remotely operated device 140. Ineither case, the modular sensing device 150 can transmit or execute thecommands accordingly (225).

At any given time in any given mode, the modular sensing device 150 canidentify a sub-mode trigger 186 in the final state machine reports 188(210). In response to the sub-mode trigger 186, the modular sensingdevice 150 can reconfigure interpretation of one or more final reportsfrom one or more corresponding state machines (215). Based on each finalstate machine report 188, the modular sensing device 150 can generatecommands in accordance with the mode and the sub-mode of the device 150(220), including the altered interpretations based on the sub-modetrigger(s) 186. As discussed above, such generated commands can includeoutput commands 164 to output audio (222), visual (226), and/or haptic(224) output on the modular sensing device 150. Additionally oralternatively, the modular sensing device 150 can generate controlcommands 162 (228) for operating a remotely operated device 140. Ineither case, the modular sensing device 150 can transmit or execute thecommands accordingly (225).

FIG. 2B is a flow chart describing a state machine-based example methodof controlling a remotely operated device using a modular sensingdevice. In the below discussion of FIG. 2B, reference may be made toreference characters representing like features as shown and describedwith respect to FIGS. 1A and 1B. Furthermore, the methods and processesdescribed with respect to FIG. 2B may be performed by an example modularsensing device 150 as shown and described with respect to FIG. 1B. Stillfurther, the steps described in FIG. 2B represent steps performed toproduce a palm control illusion (230) in the drive mode, as describedherein. Referring to FIG. 2B, the modular sensing device 150 can detecta mode selection to control a remotely operated device 140 (e.g., adrive mode) (235). The mode selection can be based on an input on themode selector 155 (236), or based on a specified gesture performed bythe user (e.g., a push gesture with the wearable device) (238).

In response to the mode selection, the modular sensing device 150 canreconfigure state machine report interpretation corresponding to usergestures for controlling the remotely operated device 140 (240). Assuch, final state machine reports 188 can be interpreted by the modularsensing device 150 as control commands 162 for operating the remotedevice 140. Furthermore, in certain implementations, the modular sensingdevice 150 can correlate sensor data thresholds to individual statemachines for switching between sub-modes (245). Specifically, the finalstate reports 188 from those state machines can comprise sub-modetriggers 186 indicating that a gesture or combination of gestures hasbeen performed (e.g., raising an arm above a threshold angle) thatcauses the modular sensing device 150 to reconfigure its interpretationof certain final state reports 188.

When initializing the drive mode, the modular sensing device 150 canbegin in an initial drive sub-mode (250), which enables initial ordefault control of the remote device 140 using a specified set ofgestures. In the initial drive sub-mode, the modular sensing device 150can dynamically generate control commands 162 for the remote device 140based on final state reports 188 received from the state machines (255).Such commands 162 can include forward acceleration commands (262),directional or steering commands (263), and/or deceleration or reversecommands (264). Furthermore, the initial drive sub-mode (250) cancorrespond to user control of the remote device 140 moving forward andaway from the user. After generating each command 162, the modularsensing device 150 can transmit the control command 162 to the remotedevice 140 for execution (260). Accordingly, the user can control theremote device 140 using arm gestures (or other gestures depending on thewearable device), causing the remote device 140 to accelerate andmaneuver accordingly. In many aspects, this initial drive mode (250) canbe perceived as the user “pushing” the remote device 140, and steeringthe remote device while performing a pushing gesture.

During operation, the modular sensing device 150 can detect a finalstate report 188 from a specified state machine correlated to a sensordata threshold (265). As described herein, this final state report 188can comprise a sub-mode trigger 186 indicating that the user hasperformed a threshold gesture. In certain examples, the thresholdgesture can correspond to the user raising and/or lowering an arm acrossa threshold angle (e.g., 45 degrees). Additionally or alternatively, thethreshold gesture can correspond to the user turning a palm fromdownwards to upwards. Furthermore, in response to the detected sub-modetrigger 186, the modular sensing device 150 can reconfigure certainfinal state report 188 interpretations in an illusion drive sub-mode(270). In certain aspects, the sub-mode trigger 186 causes the modularsensing device 150 to begin interpreting a new set of final statereports 188 that were previously not relevant in the initial drivesub-mode (250). Additionally or alternatively, interpretation of certainfinal state reports may be switched. For example, interpretation of thefinal state reports 188 for left turns versus right turns may beswitched.

Thus, the modular sensing device 150 can alter interpretation of finalstate reports 188 in generating control commands 162 for the remotedevice 140 (275). For example, the modular sensing device 150 canutilize a different final state report 188 from a different statemachine to generate forward acceleration commands for the remote device140 (276). Also, the directional or steering commands can be triggeredby different final state reports 188 as compared to the initial drivesub-mode (250). In one example, the modular sensing device 150 canswitch the state machines corresponding to such directional steeringcommands. Thus, a leftward arm movement in the illusion drive sub-mode(270) can correspond to a right turn by the remote device 140. Thus, asthe remote device 140 travels back towards the user, an illusion isperceived that the user's leftward arm movement directly correlates to aforce pulling the remote device 140 leftward from the user'sperspective. Still further, the modular sensing device 150 can alter aninterpretation of a particular final state report 188 from a statemachine to generate deceleration or reverse commands (278).

When such gestures are completed by the user wearing the modular sensingdevice 150, the modular sensing device can dynamically generate andtransmit the control commands 162 to the remote device 140 for execution(280). Thus, as long as another threshold gesture is not detected, theuser can operate the remote device 140 in the illusion drive sub-mode(270), which can be perceived as the user “pulling” the remote deviceand steering the remote device in conjunction with the pulling armgesture. While in the illusion drive mode (270), the modular sensingdevice 150 can receive a final state report 188 indicating anothersensor data threshold (285). In some examples, the threshold cancorrespond to crossing the previous threshold in a reverse manner (e.g.,going from palm up to palm down and/or lowering the arm below 45 degreesin relation to the horizontal plane). In variations, the drive mode caninclude a third, fourth, fifth, or n drive sub-mode in the underlyingdrive mode, in which the modular sensing device 150 reconfiguresinterpretation of certain final state reports 188, or begins tointerpret new final state reports 188. In any case, upon detecting thefinal state report 188 indicating the respective sensor data threshold,the modular sensing device 150 can revert to the initial drive sub-mode(250), or a separate drive sub-mode depending on the gesture threshold.

In certain implementations, some control commands 162 can be attributedto two or more state machines. For example, a first state machine can beassociated with an acceleration command in the drive mode, and a secondstate machine can also be associated with the acceleration command.Thus, final state reports 188 from each of these state machines cancause the modular sensing device 150 to generate an accelerationcommand. Along these lines, two or more state machines can also causethe modular sensing device 150 to generate the same output via thehaptic 192, visual 194, and audio 196 systems in the same mode. Forexample, swinging gestures may be performed from multiple relativeangles (e.g., corresponding to tennis swings or sword fighting strokes).In certain modes, final state reports 188 from state machinescorresponding to each of such swings can cause the modular sensingdevice 150 to output the same or similar sounds, vibrations, and/orlight outputs.

Modular Sensing Device

FIG. 3 illustrates an example modular sensing device insertable into aplurality of compatible apparatuses. The modular sensing device 300shown in FIG. 3 can comprise various components and modules of themodular sensing devices 100, 150 shown in the block diagrams of FIGS. 1Aand 1B. Referring to FIG. 3, the modular sensing device 300 can includea number of output devices, such as an LED array 310, an audio outputdevice 320 (e.g., a speaker), and a haptic driver 360 (included withinthe device). Furthermore, the modular sensing device 300 can include amode selector 330, which can comprise an analog or digital button toenable the user to select a particular mode of the device 300 by, forexample, scrolling through a stored series of modes. The modular sensingdevice 300 can further include memory and processing resources 365 thatcan execute the selected mode (either in the state machineimplementation (FIG. 1B) or the executed instruction set implementation(FIG. 1A) described herein).

In various aspects, the modular sensing device 300 also includes acommunications interface 370 (e.g., a BLUETOOTH low energy, WiFi, WiGig,WiMAX, or cellular radio interface), and an IMU 340 to provide thememory and processing resources 365 with sensor data for detectinggestures performed by the user. As described herein, depending on themode and sub-mode of the device 300 the memory and processing resources365 interpret the sensor data to generate outputs via the output devices310, 320, 360 and/or commands or responses to be output to a connecteddevice via the communications interface 370 (e.g., a remotely operateddevice or another modular sensing device). Furthermore, in someimplementations, the modular sensing device 300 can include an inputinterface 350 (e.g., a mini-USB port) to enable charging of one or morebatteries and/or uploading of additional mode instructions. Invariations, the modular sensing device 300 can include an inductioninterface to charge one or more batteries and/or to enable inductivepairing with a second device to establish a wireless communicationslink.

In the various examples described herein, the modular sensing device 300can be insertable into or otherwise attachable to any number ofcompatible apparatuses, such as wearable devices 395 (wrist devices,rings, pendants, hats, glasses, etc.) wielded devices 385, companiontoys or dolls, and the like. Furthermore, the modular sensing device 300can be implemented in various other form factors, can be sewn intoclothing, or can be mounted, glued, or otherwise attached to variousapparatuses. Such apparatuses can each include a module holder 387, 397into which the modular sensing device 300 may be inserted or otherwisemounted or attached. Thus, according to examples provided herein, theuser can utilize the apparatuses into which the modular sensing device300 has been inserted or attached, to perform various gestures inaccordance with a selected mode of the modular sensing device 300.

Multi-Device Usage

FIG. 4 illustrates an embodiment of multiple sensing devices thatconcurrently provide input for a program or application which utilizesthe inputs, along with inferences which can be made about a person orobject that carries the devices, according to one or more examples. Inparticular, an example such as shown enables input from multiple sensingdevices to be used for purpose of enabling inferences of movement andpose from two relevant sources of user motion. For example, in FIG. 4, auser 401 carries wearable devices in the form of a wrist device 410 andpendent 412. In other examples, one or both of the wrist device 410 andpendent 412 can be in the form of an alternative form factor or devicetype. For example, the combination of sensing devices can include a hat,a ring, eyeglasses or a device which the user can carry in his or herhand (e.g., FOB, mobile computing device). In variations, more than twowearable devices can be employed by one user.

FIG. 5 illustrates a system which concurrently utilizes input frommultiple modular sensing devices in connection with execution of anapplication or program. With reference to an example of FIG. 5, amulti-device system includes a first modular sensing device 510, asecond modular sensing device 520, and a controller 530. Each of thefirst and second modular sensing devices 510, 520 includes a respectiveinertial measurement unit (IMU) 512, 522, a processor 514, 524 andmemory 516, 526. The IMU 512, 522 of each modular sensing device 510,520 can include sensors such as an accelerometer 515, 525 and gyroscopicsensor 517, 527. The first and second modular sensing devices 510, 520may also include additional sensing resources, such as a magnetometerand/or proximity sensor.

The controller 530 can include a processor 532 and a memory 534. Theprocessor 532 can execute instructions 535 for a program or applicationthat can execute and process inputs 511, 513 from each of the respectivemodular sensing devices 510, 520. In some variations, the controller 530is a mobile computing device, such as a multi-purpose wirelesscommunication device which can wirelessly communicate with each of thefirst and second modular sensing devices 510, 520.

While an example of FIG. 5 illustrates the controller 530 as a separatedevice from the first and second modular sensing devices 510, 520,variations provide that the controller 530 is integrated or otherwisecombined with at least one of the first or second modular sensingdevices 510, 520. For example, the controller 530 can include amulti-purpose wireless communication device that is equipped with agyroscopic sensor and accelerometer. Thus, for example, variations canprovide the second modular sensing device 520 to be a local resource ofthe controller 530, which communicates with the first modular sensingdevice 510.

With further reference to FIG. 5, the controller 530 can receive inputs511, 513 from respective first and second modular sensing devices 510,520. The inputs 511, 513 can be received in connection with anapplication 539 or program that is executed by the processor 532 of thecontroller 530. The processor 532 can execute the instructions 545 inorder to implement an inference engine 535 for determining inferencesabout the person or object with one or both of the modular sensingdevices 510, 520. For example, the application 539 can correspond to agame or simulation, and the inference engine 535 can be specific to theapplication 539. Among other applications, the inference engine 535 canbe used to determine when the motions of two modular sensing devices510, 520 are separate and distinct from one another, or continuousand/or part of the same input motion.

According to one implementation, each input 511, 513 can correspond toone or more of a position, height, orientation, velocity, linear and/orrotational acceleration. Each of the first and second sensing devices510, 520 generate a set of measured (or sensed data) corresponding to,for example, a movement (e.g., gesture) made with the respective sensingdevice 510, 520. Additionally, the controller 530 can process input 511,513 corresponding to each of the respective data sets in order todetermine a third data set of inferences. In this way, the inferencesreflect information determined from sensed data, rather than directlymeasured data. The inferences which can be output from the inferenceengine 535 can be determinative or probabilistic, depending onimplementation.

With reference to an example of FIG. 4, user 401 can wear two modularsensing devices, and the inference engine 535 can assume some inferencesbased on anatomical constraints and/or context (e.g., such as providedfrom execution of the application 539). For example, each of the firstand second modular sensing devices 510, 520 can correspond to a wearablewrist device. Alternatively, the second modular sensing device 520 cancorrespond to the pendent 412 or neck-worn device. By way of example, ifthe first modular sensing device 510 (wrist device 410) is detected tobe in motion, the inference engine 535 can be used to determineadditional position data for the movement of that device along a thirdaxis based on orientation, position or context of second modular sensingdevice 520 (wrist device 411 or pendent device 412). For example, if thefirst modular sensing device 510 (wrist device 411) measures arc motion,and the second modular sensing 520 is the pendent, then the orientationof the second modular sensing device can indicate whether, for example,the arc motion is in front of the user or to the user's side.Alternatively, if the second modular sensing device 520 is the secondwrist device 412, the information sensed from the second wrist devicecan identify the corresponding hand or device as being in front of thebody. In such an orientation, the inference engine 535 can determine theinference to be that the user is making the arc of motion in front ofhis body. Similarly, if the height of the second sensing device 520 isdetermined to be belt high and the device is held by the user, theorientation of the user's torso can be inferred (along with thedirection of the arc).

In examples in which the second modular sensing device 520 is a pocketdevice (e.g., mobile computing device, FOB), information can bedetermined from, for example, the height of the device (e.g., userstanding, crouching or jumping) and the rotation of the device. Forexample, if the second modular sensing device 520 is pocket word, achange in the orientation of the device from vertical to horizontal, incombination with a downward acceleration can indicate the user iscrouching. If the user is crouching, for example, the type of motionthat is likely by the first modular sensing device 510 may be limited(e.g., motion of the wrist device 410 is likely in front of user whenuser is moving up or down). The examples described with respect to FIGS.4 and 5 can enable the user to utilize the modular sensing device(s) inconnection with a real-world gameplay environment (or othertask-oriented activities) executed by one or more of the modular sensingdevices 510, 520, control a remotely operated device using gesturessensed by the modular sensing devices 510, 520, interact with otherusers, and perform various tasks in which the modular sensing devices510, 520 can provide feedback and response output.

Hardware Diagram

FIG. 6 is a block diagram of an example modular sensing device uponwhich examples described herein may be implemented, such as the wearabledevice 100 of FIG. 1A.

In an example of FIG. 6, the modular sensing device 600 includes aprocessor 610, memory resources 620, a feedback mechanism 630 (e.g.,audio 632, haptic 633, visual 631 devices), a communication sub-systems640 (e.g., wireless communication sub-systems such as BLUETOOTH lowenergy), one or more sensors 660 (e.g., a gyroscopic sensor oraccelerometer) and an input mechanism 650 (e.g., an analog or digitalmode selector). In one example, the communication sub-system 640 sendsand receives data over one or more channels.

The memory resources 620 can store mode instructions 623 correspondingto a plurality of control modes 622, as described herein, which can beexecuted by the processor 610 to initiate a particular mode. Certainexecuting mode instructions 623 can initiate the communicationsub-system 640 to establish one or more wireless communication linkswith the self-propelled device and/or the mobile computing device.Execution of a control mode 622 by the processor 610 may cause theprocessor 610 to generate distinct feedback responses using the feedbackmechanism 630 based on sensor data from the sensor(s) 660 indicatinguser gestures performed by the user.

In some examples, the memory resources 620 can comprise a number ofstate machines 624 which can provide state machine reports 627 to theprocessor 610 can specified sensor patterns are identified by respectivestates machines 624. Each state machine 624 may monitor for a singlesensor pattern which, if identified by that state machine 624, can causethe state machine 624 to transition states, thereby providing a statemachine report 627 to the processor 610 identifying the user gestureperformed. The processor 610 can translate the state machine reports627—which indicate the user gestures—in accordance with an executing setof mode instructions 623 in order to generate a corresponding output viathe feedback mechanism 630 and/or control commands 612 to becommunicated to the self-propelled device via the communicationsub-system 640.

CONCLUSION

It is contemplated for examples described herein to extend to individualelements and concepts described herein, independently of other concepts,ideas or system, as well as for examples to include combinations ofelements recited anywhere in this application. Although examples aredescribed in detail herein with reference to the accompanying drawings,it is to be understood that this disclosure is not limited to thoseprecise examples. As such, many modifications and variations will beapparent to practitioners skilled in this art. Accordingly, it isintended that the scope of this disclosure be defined by the followingclaims and their equivalents. Furthermore, it is contemplated that aparticular feature described either individually or as part of anexample can be combined with other individually described features, orparts of other examples, even if the other features and examples make nomentioned of the particular feature. Thus, the absence of describingcombinations should not preclude the inventor from claiming rights tosuch combinations.

What is claimed is:
 1. A modular sensing device comprising: an inertialmeasurement unit to generate sensor data corresponding to user gesturesperformed by a user; a mode selector enabling the user to select a modeof the modular sensing device out of a plurality of modes; one or moreoutput devices to generate output based on the user gestures and theselected mode; and a controller to implement a plurality of statemachines, each respective one of the plurality of state machines beingassociated with a corresponding user gesture by a sensor data signature,wherein the respective state machine executes a state transition whenthe sensor data matches the sensor data signature, the executed statetransition causing the controller to generate a corresponding output viathe one or more output devices specific to the selected mode and basedon the corresponding user gesture.
 2. The modular sensing device ofclaim 1, wherein the respective state machine continuously iteratesbased on the sensor data from the inertial measurement unit andautomatically executes the state transition when the sensor data matchesthe sensor data signature.
 3. The modular sensing device of claim 1,wherein the modular sensing device is insertable into a wearable device,and wherein the corresponding user gesture comprises a sequence ofactions performed by the user utilizing the wearable device.
 4. Themodular sensing device of claim 1, wherein the state transition of therespective state machine causes the controller to generate distinctoutputs based on the selected mode.
 5. The modular sensing device ofclaim 1, wherein the corresponding output comprises one or more controlcommands enabling the user to control a remote controlled device usingthe modular sensing device.
 6. The modular sensing device of claim 1,wherein the one or more output devices include a haptic system, an audiosystem, and a visual system.
 7. The modular sensing device of claim 4,wherein the controller initiates a sub-mode of the modular sensingdevice when the user performs a particular user gesture, the sub-modecausing the controller to modify interpretation of the state transitionfor the selected mode, and wherein the modular sensing device generatesthe corresponding output based on the modified interpretation.
 8. Themodular sensing device of claim 3, wherein the sensor data signatureprovides the respective state machine with a threshold error tolerancefor each of the sequence of actions.
 9. The modular sensing device ofclaim 1, wherein the respective state machine comprises a finite set ofstates, one or more inputs, one or more outputs, a transition function,and an initial state.
 10. The modular sensing device of claim 1, whereinthe respective state machine comprises a set of states, and wherein eachstate in the set of states transitions in sequential order in order tocause the state transition.
 11. The modular sensing device of claim 3,wherein, for a particular iteration, the respective state machineterminates when the corresponding user gesture diverges from thesequence of actions.
 12. The modular sensing device of claim 3, wherein,for a particular iteration, the respective state machine automaticallyterminates if one of the set of states does not transition.
 13. Themodular sensing device of claim 1, wherein the controller initiates asecond mode, from the plurality of modes of the modular sensing device,when the user performs a specific user gesture.
 14. The modular sensingdevice of claim 1, wherein the modular sensing device is configured tobe portable and insertable into at least one compatible apparatus. 15.The modular sensing device of claim 14, wherein the at least onecompatible apparatus comprises a wrist-worn device and a wielded device.16. A method of generating outputs based on user gestures, the methodbeing performed by a modular sensing device and comprising: receiving amode selection input to select a mode from a plurality of selectablemodes; in response to the mode selection input, utilize a plurality ofstate machines to interpret user gestures in accordance with theselected mode, each respective one of the plurality of state machinesbeing associated with a corresponding user gesture by a sensor datasignature, wherein the respective state machine executes a statetransition when the sensor data matches the sensor data signature; andin response to the executed state transition, generating a correspondingoutput via one or more output devices of the modular sensing devicespecific to the selected mode and based on the corresponding usergesture.
 17. The method of claim 16, wherein the selected mode comprisesa drive mode, the drive mode enabling the user to control aself-propelled device with the modular sensing device.
 18. The method ofclaim 16, wherein the selected mode comprises a smart home control mode,the smart home control mode enabling the user to control at least onesmart home device using the modular sensing device.
 19. The method ofclaim 16, wherein the selected mode comprises a gaming mode, the gamingmode causing the modular sensing device to output task-orientedinstructions for the user to perform in connection with a game.
 20. Themethod of claim 16, wherein the one or more output devices include ahaptic driver, one or more light elements, and one or more audioelements.